Stability of Tsallis entropy and instabilities of 
Renyi and normalized Tsallis entropies: 
A basis for ^-exponential distributions 
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> The ^-exponential distributions, which are generalizations of the Zipf-Mandelbrot 

00 

O power-law distribution, are frequently encountered in complex systems at their 

\Q 

O 

CN stationary states. From the viewpoint of the principle of maximum entropy, they can 

2 apparently be derived from three different generalized entropies: the Renyi entropy, the 



Tsallis entropy, and the normalized Tsallis entropy. Accordingly, mere fittings of 
observed data by the ^-exponential distributions do not lead to identification of the 
correct physical entropy. Here, stabilities of these entropies, i.e., their behaviors under 
arbitrary small deformation of a distribution, are examined. It is shown that, among the 
three, the Tsallis entropy is stable and can provide an entropic basis for the q- 
exponential distributions, whereas the others are unstable and cannot represent any 
experimentally observable quantities. 
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I. 



INTRODUCTION 



It is known [1-3] that there are a number of complex systems whose statistical 
properties at the stationary states are well described by the ^-exponential distributions, 
which are generalizations of the Zipf-Mandelbrot power- law distribution [4]. The q- 
exponential distributions are anomalous distributions from the viewpoint of 
conventional statistical mechanics characterized by Boltzmann's exponential factor. 
Since so frequently observed in nature, it is of importance to develop bases for such 
distributions. In this context, we wish to mention that quite recently the ^-exponential 
factor has been obtained for the logistic map at the edge of chaos by the renormalization 
group method as well as by the Pesin equality for the generalized Kolmogorov-Sinai 
entropy and the generalized Lyapunov exponent [5]. There, the value of the entropic 
index has been calculated analytically. 

The explicit form of the ^-exponential distribution is the following: 




(i = l,2,-,W), 



(1) 



w 



z ? a)=I> 9 (-Ae,), 



(2) 



where W is the number of accessible microscopic states of a system under 
consideration, Q i the z'th value of a physical quantity Q, X a factor related to the 
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Lagrange multiplier, and e q {t) the ^-exponential function defined by 



[l + (l-4)?] 1/(1 - 9) (l + (l-q)t>0) 



(l + (l-^<0) 



(3) 



q is a positive real number termed the entropic index. This distribution has the cut-off at 
2;, max = 1/[(1 _< 7)^] if < q < 1, whereas it is equivalent to the Zipf-Mandelbrot-type 
asymptotic power- law distribution with the exponent l/(g-l) if q>\. In the limit 
q — > 1, the ^-exponential function converges to the ordinary exponential function and so 
does the ^-exponential distribution to the Boltzmann-Gibbs-Jaynes exponential 
distribution. 

Following Gibbs' procedure, one may also wish to derive the ^-exponential 
distribution from the stationarity condition on a certain generalized entropy. Such an 
entropy is found to be not unique, however. There exist three known different entropies 
that are maximized by the ^-exponential distribution under the constraint on the 
normalized ^-expectation value of Q. This can be seen as follows. 



Consider the functional 



f w \ ( w \ 

& J) [p;a,P] = Sl J) [p]-a 2> ; -l -j8 £^G i -e 9 

Vi=l J Vi=l J 

(J = R, T, NT). 



(4) 



a and /5 are the Lagrange multipliers associated with the normalization condition on 
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the basic distribution, {p J i=h2 ,-,w> an< ^ tne normalized ^-expectation value of Q, 
P; <2, = Q q , where P j is the escort distribution [6] defined by 



P = 



(p,r 



i T-i W 



(5) 



The three generalized entropies are listed as follows: 



i w 

s ( q R) [p] = -^-^( Pi y, 



(6) 



l-q 



X>,r-i 



(7) 



Si m [p] = 



l-q 



1- 



(8) 



which are the Renyi entropy [7], the Tsallis entropy [8], and the normalized Tsallis 
entropy [9,10], respectively. These are connected to each other in the obvious ways, and 
all converges to the Boltzmann-Gibbs-Shannon entropy in the limit q — > 1: 



lim [p] = lim 5< r) [p] = lim T) [p] = S[p] = -V Pi lnp r (9) 



For a statistically independent bipartite system, (A, B), these entropies satisfy 




(A, B) 




(A) + 




{B) + T (J \q)S ( q J \A)S^{B), 



(10) 



where 



T ( ^) = 0, 



(ID 



(12) 



r (NT \q) = q-l. 



(13) 



Thus, the Renyi entropy is additive, whereas the Tsallis and normalized Tsallis 
entropies are nonadditive. 

The Renyi entropy is conventionally used for the definition of the generalized 
dimension in multifractals [6], and the Tsallis entropy plays a central role in 
nonextensive statistical mechanics [2-4]. 

Variation of <t> (/) with respect to p i gives rise to the following stationary 
distribution: 




1 



(14) 



w 




(15) 



where X ' s are given by 
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X (T) = 



(17) 



X (NT) 




(18) 



respectively, provided that, in Eqs. (17) and (18), we have put 



w 




I>, (/) r- 



(19) 



Also, in the above expressions of A's, Q ( q J) stands for the normalized ^-expectation 



value of Q with respect to p\ ' in Eq. (14). Here, it is worth mentioning that, as long as 
the ^-exponential distribution is concerned, the expectation value has to be defined in 
terms of the escort distribution as in Eq. (4), since only in this case the principle of 
maximum generalized entropy can be consistent with the principle of equal a priori 
probability [11]. 

Thus, in fact, the Renyi, Tsallis, and normalized Tsallis entropies all lead to the q- 
exponential distributions of the same type. In other words, mere fittings of observed 
data by the q-exponential distributions do not tell us anything about which the 
underlying physical entropy is. In this respect, it should be noted that, in Ref. [12], 
Lesche has presented a counterexample showing instability of the Renyi entropy. 



In this paper, we show that the Renyi and normalized Tsallis entropies are unstable 
under small deformation of a distribution and therefore cannot represent experimentally 
observable quantities, whereas the Tsallis entropy is stable and can provide an entropic 
basis for the ^-exponential distributions. The discussion is general and is independent of 
any stationary properties. 

The paper is organized as follows. In Sec. II, the rigorous definition of stability of a 
statistical quantity is given. In Sec. Ill, instability of the normalized Tsallis entropy as 
well as the Renyi entropy is shown. In Sec. IV, a general proof is established for 
stability of the Tsallis entropy. Sec. V is devoted to conclusion. 



II. OBSERVABILITY AND STABILITY 

Consider a statistical quantity C - C[p], which has its maximum value, C max . 
C[p] is said to be stable if the amount of its change under an arbitrary small 
deformation of the distribution remains small. Any observable quantities have to be 
stable, since otherwise their values cannot be experimentally reproducible. Let us 
measure the size of deformation from {p i } ;=1 2 ... w to [p , } I=1 2 ... w by the / 1 - norm: 

w 

\\p-p'\\,=Y,\Pi-p'i\- ( 2 °) 
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Note that this quantity should be independent of W. Then, an observable quantity, C[p], 



has to possess the following property [12]: 



(V£>0) (38 >0) 



p-p'W^S 



C[p]-C[p'] 



< e 



(21) 



for arbitrary values of W. 



III. INSTABILITY OF RENYI AND NORMALIZED TSALLIS ENTROPIES 

IN THE THERMODYNAMIC LIMIT 



In this section, we discuss instabilities of the Renyi and normalized Tsallis entropies 
by using a counterexample which violates the condition in Eq. (21). 

First of all, we recall that Renyi, Tsallis, and normalized Tsallis entropies take their 
maximum values for the equiprobability p i = 1 / W (z = 1, 2, • ••, W) : 

S ( «L = lnW, (22) 
Sj!L=ln,W, (23) 
0=-ln,W- 1 . (24) 



Here, In x stands for the ^-logarithmic function defined by 



In i = — (/"'-l) (;c>0) 
q l-q 



(25) 



which is the inverse function of the ^-exponential function and converges to the 
ordinary logarithmic function in the limit q—>l. 

The deformation of a distribution to be examined is given as follows [12]: 



• < q < 1; 



{ 8 W 

v 2 W-l 



\ 



Pi + 



S 1 
2 W-l 



(26) 



q>l; 



1 



(27) 



Clearly this preserves the normalization condition. In both the cases of < q < 1 and 
g>l,the /' -norm is seen to be 



p-p\\=8. 



(28) 



Also, from Eqs. (26) and (27), it is immediate to obtain 



• < q < 1; 

w 



1- 


<5^ 


9 


^ .1 




+ 




V 


2; 




K2) 



(W-l) 



l-q 



(29) 



q>U 



yy yy 



5^ 



+ 



v ±) 



(W-l) 



l-q 



(30) 



a. Renyi entropy 

The following discussion about instability of the Renyi entropy can be found in Ref. 
[12], but we present it here in order to make the discussion self-contained. 
Using Eqs. (29) and (30) in Eq. (6), we find: 



0<q<l; 



1 



S ( q K, [p] = 0, S?W = - In 

l-q 

1 





q 




+ 




I 1) 




K2) 



(W-l) 1 " 9 



q, max 



l-q 



In 



1- 



V 



+ 



2 J \2) 



(W-l) 



l-q 



->1 



InW 

(W_>oo). 



(31) 



(32) 



q>l; 



S^[p] = ln(W-l), 



1 



s™[pl-s?W 



q, max 



Sl K> [p'] = — In 
1-g 

1 



f 5> 


9 


1- 


<5^ 




+ 




v2; 






2J 



(W-l) 



l-q 



(33) 



ln(W-l) 



In 



(8^ 








+ 




v2j 




V 2j 



(W-l) 



1-9 



InW 



-> 1 



(VK^oo). 



(34) 
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Therefore, the condition in Eq. (21) is violated. 



b. Tsallis entropy 



Using Eqs. (29) and (30) in Eq. (7), we find: 



0<q<l; 



Sl T) [p] = 0, Sl T) [p<] = 



l-q 



f R\ 



( RX 



+ 



(W-l) 1 " 9 -! 



(35) 



AT) 
q, max 



( sy fsy 



+ 



(w-iy- q -i 



W l ~ q - 1 



f XV 



— > 



(W->oo). 



(36) 



q>\; 



S^[p] = ln q (W-l), 5 J r) [p , ] = — 



f 5> 




1- 






+ 




v2; 




v. 


2) 



(w-iy- q -i 



, (37) 



Sl T) [p]-Sl'>[p'] 



(T) r 



AT) 
q, max 



8\ 



Ml 



(W-l) 



1-9 



'5Y 



(W_»oo). 



(38) 



Therefore, if (5 is taken to be (5 < 2e llq , the condition in Eq. (21) is satisfied. 
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c. Normalized Tsallis entropy 



Using Eqs. (29) and (30) in Eq. (8), we find: 



• < q < 1; 



l-q 



1- 



Ml 



+ 



2) 



(W-l) 



,(NT) 



( 

1- 


5> 








+ 




V 


2) 




K2) 



(W-l) 1 " 9 



-> 1 



l-w 9 " 1 



(39) 



(40) 



Sl NT) [p] = -ln q (W-iy\ 



l-q 



1- 







i- 


5 1 




+ 










2J 



, (41) 



(W-l) 1-9 



J q, max 





+ 


( 

1- 

V 


<5^ 

2) 


q 

(W-l) 1 " 9 


1- 


-W 


9-1 







^1 (^/^oo). 



(42) 



Therefore, as in the case of the Renyi entropy, the condition in Eq. (21) is violated. 
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The results in Eqs. (32), (34), (40), and (42) mean that the Renyi and normalized 
Tsallis entropies with < q < 1 overestimate a large number of occupied states even if 
their overall probability is so small that they are irrelevant and those with q > 1 
overestimate a high peak of probability. 

Thus, among the three, there is a possibility only for the Tsallis entropy to be 
observable. 



Stability of the Boltzmann-Gibbs-Shannon entropy has been shown in Ref. [12]. 
Here, we prove stability of the Tsallis entropy by generalizing the discussion in Ref. 



IV. 



STABILITY OF TSALLIS ENTROPY 



[12]. 



Let us define the following quantity: 



w f 

\[p;t) = Y, Pi 



e q (t) 



1 



(43) 



where tis a positive parameter and the symbol (x) + means 



(x) + = max}*, 0}. 



(44) 



The following will be useful later: 
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(x) + = x6(x), 

I (x) + -(y) + \ <\x-y\, 



(45) 
(46) 



where 0{x) is the Heaviside unit step function defined by 0{x) - for x < and 
6{x) = \ for x>0. 

The quantity in Eq. (43) has several interesting properties. 

From Eq. (46), it immediately follows that 



\A q [p;t)-A q [p';t)\<\\p-p'\\ v 
Using the relation 



( »' 


[""«.« 




w 


( 


1 \ 


E 




^E 

i=i 

+ 






) 


Pi 

V 





<1, 



we have 



<A q [p;t)<h 



In particular, if t > \n g W , then Eq. (49) becomes 



i-^\u,»<i. 



(47) 



(48) 



(49) 



(50) 
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The same is true for another distribution [p i } I=1 2 ... w , that is, 



-K-A \p';t)<-l + 



W 

e q (t) 



(51) 



Adding Eqs. (50) and (51), we find 



W 



A q [p;t)-A q [p';t) < — (Vf>ln,W) 



e q (t) 



(52) 



In the limit t t m . dX with ? max =°° (0<q<\), l/(q-l) (q>\) [see Eq. (3)], 
e q (t) diverges and therefore A q [p;t) tends to unity. So, the integral 

j" m ™ dt (\ - A q [p;t)} may converge. This is in fact the case. Using Eq. (45), this integral 



is written as 



\ dt(l-A q [p;t)) = £ \ dt 



i=l o 



1-0 



* max 

+W f dt . 

J e(t) 



(53) 



The second term on the right-hand side gives 



W j dt 



1 W 



e q (t) q 



(54) 



On the other hand, noting < ln ? (l / p t ) < r max , the integral in the first term is 
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calculated as follows: 



max 



( . \ 



1-0 



Pi 



;r to,(l/p ( ) f "> 



= 1 I * 



( = 1 o 



1-9 



£ (p,-) 9 -i 



1 _!L 



(55) 



Therefore, we obtain 



' max 

J ^(l-A 9 [p;0) = -Sf[p] + - 



(56) 



or, conversely, 



• max 

Sj r) [p] = -1 + <Z J ^(l-A,[p;0). 



(57) 



Now, using the representation in Eq. (57) and taking a satisfying -In W <0<a 



<U>we have 



S™[p]-S™[p]\ = q 



j ^(A 9 [p;0-A 9 [/y;0) 
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<q \ dt\A q [p;t)-A q [p';t) 



a+ln„ W 



= q J" dt\A q [p;t)-A q [p';t)\ 

o 

' max 

+ q J J? | A 9 [p;0-A ? [p';0 |. (58) 



fl + ln„ W 



From Eq. (47), the first integral is found to satisfy 



a+ln „ W 



J" dt\A q [p;t)-A q [p';t)\<\\p-p'\\ 1 (a + \n q W). (59) 



Likewise, from Eq. (52), the second integral is evaluated as 



max max „ , „ , _ 

\ dt\A q [p;t)-A q [p';t)\< J rff — ^[^(fl + ln^)]"'. (60) 

a+\n„W a+\n„W £ ? 



Therefore, we have 



C [p] - [p 1 ] | < ? II p - p' ll t (a + ln 9 W)+ — . (61) 

h( a+ln * w )J 



This inequality holds for any values of a satisfying -ln ? W < < a < t m . dX . Evaluating 
the minimum of the right-hand, which is realized when 



a + ln a W = ln a , (62) 

q q \\p-p'\\ l 
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Eq. (57) is reexpressed as follows: 



W 



S { q ' > [p]-Sl J, [p']\<q\\p-p'\\ 1 ln " +W l - q (Wp-p'\\ i y. (63) 
1 \\p- p \\ l 



Using the equality, In (y / x) = x q 1 (In y - In x) , we further obtain 



^ r) [p]-^ r) [p'] < (llp-p'll^ ln ? W 



+ (llp-p'll 1 ) 9 (l-^ln II />-/>' l^), 



(64) 



from which we find 



c(r) 

'-'ij, max 



< 



(H 



Hi) + : 



(11^-^' II,) 9 (0<q<l) 



q\\p-p , \\ l (q>l) 



(65) 



Therefore, taking \\ p- p' \\ { < d < £ 1,q (0<q<l) or II p - p' \\ { < 8 < £ I q (q>l), we 
see that the condition in Eq. (21) is satisfied by the Tsallis entropy. 

The above discussion holds for \f q> 0, and so, as a simple byproduct, stability of 
the Boltzmann-Gibbs-Shannon entropy [12] corresponding to the limit g — > 1 (also of 



the Renyi and normalized Tsallis entropies) is reestablished. 
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V. CONCLUSION 



We have shown that among the Renyi, Tsallis, and normalized Tsallis entropies, only 
the Tsallis entropy is stable and can give rise to experimentally observable quantities. 
Therefore, it is the Tsallis entropy, on which the ubiquitous ^-exponential distributions 
have their basis. A remaining important (and hard) question is if the Tsallis entropy is 
the unique generalized entropy. In this respect, we wish to mention that there are some 
affirmative points: there exist a set of axioms and the uniqueness theorem for the Tsallis 
entropy [13], and the structure of nonadditivity in Eq. (10) is essential from the 
viewpoint of the zeroth law of thermodynamics [14,15]. 
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